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For the Promotion of the Mechanic Arts 


Hall of the Institute, 
Philadelphia, June 15, 1955. 


Committee on Science and 
the Arts Case No. 3290. 


t 


The Franklin Institute of the State of Pennsylvania, acting 
through its Committee on Science and the Arts, has considered carefully the 
work of those who have contributed greatly to the field of research in 
communication and reconnaissance, and has selected as recipient of the award 
of the Stuart Ballantine Medal for 1955 - 


CLAUDE ELWOOD SHANNON, 
of Chatham, New Jersey, 


In consideration of his recognition of communication 
as essentially a statistical process, his identifica- 
tion of the elements of communication systems with 
the appropriate statistical functions, and his weld- 
ing of the powerful methods of mathematical statistics 
into a comprehensive theory of communication which 
permits precise and rapid evaluation of proposed new 
communication systems, and points the way for 
significant new developments. 
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This report may be introduced by several quotations from Dr. Warren 
Weaver's paper, "Recent Contributions to the Mathematical Theory of Communication": 

"The word communication will be used here in a very broad sense to 
include all of the procedures by which one mind may affect another. This, of 
course, involves not only written and oral speech, but also music, the pictorial 
arts, the theatre, the ballet, and in fact all human behavior." ,.. 

"Relative to the broad subject of communication, there seem to be 
problems at three levels. Thus it seems reasonable to ask serially: 

"Level A. How accurately can the symbols of communication be 
transmitted? (The technical problem.) 

"Level B. How precisely do the transmitted symbols convey the 
desired meaning? (The semantic problem.) 

"Level C. How effectively does the received meaning affect conduct 
in the desired way? (The effectiveness problem.) 

"The technical problems are concerned with the accuracy of 
transference from sender to receiver of sets of symbols (written speech), or of a 
continuously varying signal (telephonic or radio transmission of voice or music), 
or of a continuously varying two-dimensional pattern (television), etc. 
Mathematically, the first involves transmission of a finite set of discrete 
symbols, the second the transmission of one continuous function of time, and the 
third the transmission of many continuous functions of time or of one continuous 
function of time and of two space coordinates. 

"The semantic problems are concerned with the identity, or 
satisfactorily close approximation, in the interpretation of meaning by the 
receiver, as compared with the intended meaning of the sender. This is a very 
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deep and involved situation, even when one deals only with the relatively simpler 
problems of communicating through speech." ... 





"The effectiveness problems are concerned with the success with 
which the meaning conveyed to the receiver leads to the desired conduct on his 
part. It may seem at first glance undesirably narrow to imply that the purpose 
of all communication is to influence the conduct of the receiver. But with any 
reasonably broad definition of conduct, it is clear that communication either 
affects conduct or is without any discernible and probable effect at all." .,, 

"So stated, one would be inclined to think that Level A isa 
relatively superficial one, involving only the engineering details of good 
design of a communication system; while B and C seem to contain most if not all 
of the philosophical content of the general problem of communication. 

"The mathematical theory of the engineering aspects of communica- 
tion, as developed chiefly by Claude Shannon at the Bell Telephone Laboratories, 
admittedly applies in the first instance only to problem A, namely, the technical 
problem of accuracy of transference of various types of signals from sender to 
receiver, Put the theory has, I think, a deep significance which proves that the 
preceding paragraph is seriously inaccurate. Part ої the significance of the 
new theory comes from the fact that levels B and C, above, can make use only of 
those signal accuracies which turn out to be possible when analyzed at Level A. 
Thus any limitations discovered in the theory at Level A necessarily &pply to 
levels B and C. But a larger part of the significance comes from the fact that 
the analysis at Level А discloses that this level overlaps the other levels more 
than one could possibly naively suspect. Thus the theory of Level A is, at least 
to a significant degree, also a theory of levels B and C." f 
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Plate I represents, symbolically, a communication system. "The 
information source selects a desired message out of a set of possible messages ..." 
+++ "The transmitter changes this message into a signal which is actually sent 
over the communication channel from the transmitter to the receiver." ... "The 
receiver is a sort of inverse transmitter, changing the transmitted signal back 
into a message, and handing this message on to the destination.” ... 

"In the process of being transmitted, it is unfortunately 
characteristic that certain things are added to the signal which were not intended 
by the information source. These unwanted additions may be distortions of sound 
(in telephony, for example) or static (in radio), or distortions in shape or 
shading of picture (television), or errors in transmission (telegraphy or 
facsimile), etc. All of these changes in the transmitted signal are called noise. 

"The kind of questions which one seeks to ask concerning such a 
communication system are: 

а. How does one measure amount of information? 

b. How does one measure the capacity of а communication channel? 

с. The action of the transmitter in changing the message into 
the signal often involves a coding process. What are the characteristics of an 
efficient coding process? And when the coding is as efficient as possible, at 
what rate can the channel convey information? 

d. What are the general characteristics of noise? How does noise 
affect the accuracy of the massage finally received at the destination? How can 
one minimize the undesirable effects of noise, and to what extent can they be 
eliminated? 

e. If the signal being transmitted is continuous (as in oral speech 
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or music) rather than being formed of discrete symbols (as in written speech, 
telegraphy, etc.), how does this affect the problem?" 

Inspired by the earlier work in mathematical theory of communication 
by Nyquist and Hartley, and by the power of the statistical methods applied by 
Norbert Wiener in "The Extrapolation, Interpolation, and Smoothing of Stationary 
Time Series," Dr. Shannon has developed the mathematical techniques necessary to 
provide quantitative answers to the above questions for present day communication 
Systems, including those employing some of the newer methods of modulation such 
ав pulse-code modulation (PCM) and pulse position modulation (PPM) which exchange 
bandwidth for signal-to-noise ratio. 

These techniques are summarized and generalized by Dr. Shannon in 
the paper "The Mathematical Theory of Communication," originally published in the 
Bell System Technical Journal in July and October 1918, and published in book 
form in 1919, together with the paper by Dr. Weaver from which the quotations 
above are taken. 

While an extended analysis of this рарег? is beyond the scope of 
this report, an attempt will be made to indicate the basis for the logical 
structure, and to define some of the terms frequently used in discussions of 
these methods. 

In "The Mathematical Theory of Communication," Dr. Shannon considers 
first discrete noiseless systems. He defines the capacity of a discrete channel 
as the limit (as T approaches infinity) of the ratio log N(T) where N(T) is the 
number of allowed signals of duration T, and demonstrates how this applies to 
familiar cases. He then shows that since a physical system, or a mathematical 
model of a physical system which produces a sequence of Symbols governed by a set 


*Dr. Shannon's paper. 
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of probsbilities is known as a Stochastic process, a discrete source of information 
is а stochastic process, and may therefore be treated by the mathematical methods 
developed for such processes. It can then be shown that stochastic processes of 
the type governed by a set of probabilities similar to those covering word 
Structure and sequencing are known as discrete Markoff processes, which have been 
Studied extensively in the literature. Dr. Shannon then points out: "Among the 
possible discrete Markoff processes there is a group with special properties of 
significance in communication theory. This special class consists of the 'ergodic! 
processes, and we shall call the corresponding sources ergodio sources. Although 
& rigorous definition of an ergodic process is somewhat involved, the general 

idea is simple. In an ergodic process every sequence produced by the process is 
the same in statistical properties. Thus the letter frequencies, digram 
frequencies, etc., obtained from particular sequences, will, as the lengths of 

the sequences increase, approach definite limits independent of the particular 
sequence." ... "Roughly, the ergodic property means statistical homogeniety." 

To define a quantity which will measure the rate at which informa- 
tion is produced in a Markoff process, it is possible to write down the 
specifications of this quantity. These specifications lead to a quantity 
He-K Py log Py Where the p,'s are the probabilities of the occurrence of 
the me events. This is precisely the form in which entropy is defined by 
Tolman and others in statistical mechanics. Quantities of this form play a major 
part in the development of information theory, as measures of information, choice, 
and uncertainty. 

To quote Dr. Shannon again: "The ratio of the entropy of a source 
to the maximum value it could have while still restricted to the same symbols 
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will be called its relative entropy. This, as will appear later, is the maximum 
compression possible when we encode into the same alphabet. One minus the relative 
entropy is the redundancy. The redundancy of ordinary English, not considering 
Statistical structures over greater distances than about eight letters, is 
roughly 50%. This means that when we write English half of what we write is 
determined by the structure of the language, and half is chosen freely." 

Further development of this logic leads to development of a 
mathematical description ої the encoding and decoding operations, and to expres- 
sions relating statistical characteristics of source and channel. 

When noise is introduced into a discrete system, it is, to quote 
again "not іп general possible to reconstruct the original message or the trans- 
mitted signal with certainty by any operation on the received signal E." The 
problem then becomes one of finding ways of transmitting the information which 
are optimal in combating noise. The average ambiguity of the received signal is 
called the equivocation, and is equal to the conditional entropy. Channel 
capacity is then determined by maximizing the difference between the source 
entropy and the equivocation: С = Мах (H(x) - ну(х)). It can be demonstrated 
that for a given channel capacity and given entropy of a discrete Source, there 
exists a coding system such that the output of the source can be transmitted 
over the channel with an arbitrarily small equivocation. An example of efficient 
coding is worked out. 

In proceeding to the case of continuous information, it is 
necessary to consider signals as ensembles of functions, which may be continuous 
distributions, but may be bandwidth or amplitude limited. It is possible to 
calculate the entropy of such ensembles, and to apply statistical methods to 
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problems of source rates, channel capacities and channel fidelities. 
Channel capacity is again equal to the maximized difference between 
source entropy and equivocation, and in this case, can be shown to be equal to 


о ines Mf ron toe le dx dy. 


As an example of the application of these methods to а practical 
problem, in a joint paper with В. M. Oliver and J. В. Pierce, entitled "The 
Philosophy of PCM" and published in November 1918, Dr. Shannon used these 
techniques in a comparison of PCM with broad band FM. The following results are 
easily derived: 

1, РОМ gives better signal-to-noise ratio than FM. 

2. Binary PCM gives high quality signals where it is just possible 
to recognise the presence of each pulse through noise and interference. 

3. PCM gives no improvement in signal-to-noise ratio in times of 
large signal or low noise. 

l. POM lends itself to time division multiplexing. 

Another application of Dr. Shannon's methods is found in his paper, 
"Communication in the Presence of Noise," published in January 1919. This paper 
is described by the following abstract: "A method for representing any communi- 
cation system geometrically. Messages and the corresponding signals are points 
in two function Spaces, and the modulation process is a mapping of one space into 
another. Using this representation, a number of results in communication theory 
are deduced concerning expansion and compression of bandwidth, and the threshold 
effect. Formulas are found for the maximum rate of transmission of binary digits 
over a system when the signal is perturbed by various kinds of noise. Some of 
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the properties of ideal systems which transmit at the maximum rate are discussed. 
The equivalent number of binary digits per second for certain information sources 
ів calculated." 

In a classified report, "A Mathematical Theory of Cryptography," 
dated September 1, 1945, Dr. Shannon applies many of these principles to an 
analysis of military secrecy systems. Plate II is a schematic of such a system. 
The relation between system structure and the amount of labor required for 
Solution is demonstrated, and an indication is given as to how a System should 
be constructed in order to require a very large amount of labor to solve it 
without a key. 

This paper has since been declassified, and was published in the 
Bell System Technical Journal in October 1911 under the title, "Communication 
Theory of Secrecy Systems." 

In addition to his work in the field of communications, Dr. 
Shannon has worked extensively in the fields of switching circuit synthesis, and 
the analysis and synthesis of automata, including examples of game-playing and 
learning machines. 

Twenty-two significant papers, of which Dr. Shannon has been 
author or co-author, have come to our attention, as well as numerous book reviews 
and informal discussions. These are listed in Appendix B. 

In summary Dr. Shannon has recognized that complete communication 
processes can be studied by statistical methods, and has chosen appropriate 
statistical functions for the elements of the systems. In many cases there is 
considerable background in the literature regarding the properties of the models 
he has selected. He has organized this material into a comprehensive theory, with 
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appropriate extensions, to permit the over-all study of complete communication 
systems of great complexity. This makes it possible to indicate methods by 
which performance for any particular application may be optimized, and permits 


rapid and precise comparison of the over-all characteristies of complex systems. 


A biographical note on Dr. Shannon is attached as Appendix A. 


* * + * * * * * * * * * * * * * 


* 





In consideration of his recognition of communication as 
essentially a statistical process, his identification of the elements of 
communication systems with the appropriate statistical functions, and his weld- 
ing of the powerful methods of mathematical statistics into a comprehensive 
theory of communication which permits precise and rapid evaluation of proposed 
new communication systems, and points the way for significant new developments, 
THE FRANKLIN INSTITUTE awards its STUART BALLANTINE MEDAL to CLAUDE ELWOOD 


SHANNON, of Chatham, New Jersey. 


President. 


Secretary. 


2nairman of the Committee on Science 
and the Arts. 
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Appendix A 
CLAUDE E. SHANNON 

CLAUDE E. SHANNON, mathematician and inventor, has been a member 
of the technical staff at Bell Telephone Laboratories since 1911. He has made a 
number of outstanding contributions to the communication field, especially in the 
mathematical theory of communication, application of Boolean algebra, oryptography, 
and computing circuits. 

Born in Petoskey, Michigan, on April 30, 1916, Dr. Shannon was 
graduated from the University of Michigan in 1936 (B.S. in E.E.). He continued his 
studies at Massachusetts Institute of Technology, where he received S.M. and Ph.D. 
degrees in 1940. While at M.I.T. he was awarded a Bowles Fellowship in 1939. The 
American Institute of Electrical Engineers granted him the Alfred Noble Prize in 
1910 and that year he won a National Research Fellowship. He used the latter at 
Princeton University. Не served with the N.D.R.C. ав a consultant in fire-control 
work from December 1910 until August 1941 when he joined Bell Laboratories. 

The Morris Liebmann Memorial Prize of the Institute of Radio 
Engineers was awarded to Dr. Shannon іп 1919 for his "original and important contri- 
butions to the theory of the transmission of information in the presence of noise." 
The following year, he was made a Fellow of that Institute. 

With М. Weaver, Dr. Shannon is the co-author of Ihe Mathematical 
Theory of Communication. In addition a number of his articles have been published 
in the Bell System Technical Journal, Proceedings of the Institute of Radio 
Engineers, Journal of Mathematics and Physics, Philosophical Magazine, and others. 

Dr. Shannon is a member of the American Mathematical Society, Sigma 
Xi, and Phi Kappa Phi. 

Dr. Shannon and his wife, the former Mary Elisabeth Moore, make 
their home in Chatham, New Jersey. 
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17. 


18. 


19. 


20. 
?1. 
22. 


?3. 
гі, 


25, 


26. 


Discussion of Dr. C. E. Shannon's papers (and reply), Great Britain, 
Ministry of Supply. Symposium on Information Theory. Royal 
Society, Purlington House, London, Sept. 26-29, 1950 

"Lattice Theory of Information," Great Britain, Ministry of Supply. 
Symposium on Information Theory. Royal Society, Burlington House, 
London, Sept. 26-29, 1950 


"Memory Requirements in a Telephone Exchange," Bell System Technical Journal, 
v. 29, July 1950 


"Programming a Computer for Playing Chess," Phil. Mag., v. ll, Mar. 1950 
"Recent Developments in Communication Theory," Electronics, v. 32, Apr. 1950 


Review of book "Description of a Relay Calculator" by the Staff of the 
Harvard Computation Laboratory, IRE Proc., v. 38, Apr. 1950 


"A Symmetrical Notation for Numbers," Am. Math. Monthly, v.57, Feb. 1950 


"Prediction and Entropy of Printed English," Bell System Technical Journal, 
v. 30, Jan. 1951 


"The Mathematical Theory of Communication," University of Illinois Press, 
1949 (Book) (By C. E. Shannon and W. Ne 


"Computers and Automataj' IRE Proc., v. ll, Oct. 1953 


| .X2————— — 
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Publications of Claude E. Shannon 
"A Symbolic 3 of Relay and Switching Circuits," A.I.E.E. Trans., v. 57, 
193 
"Mathematical Theory of the Differential Analyzer," Journal of Mathematics 
and Physics, v. 20, 1911 


"Number of Two-Terminal Series-Parallel Networks," Journal of Mathematics 
and Physics, v. 21, Aug. 1912 (By John Riordan and C. E. Shannon) 


"Transmission of Information" (Presented at IRE meeting, New York, Nov. 
19h7, pamphlet.) 


"Mathematical Theory of Communication," Bell System Technical Journal, v. 27, 
July, October 1918 


"Philosophy of PCM," IRE Proc., v. 36, Nov. 1948 (By B. M. Oliver, J. R. 
Pierce, and C. E. Shannon.) 


"Communication in the Presence of Noise," IRE Proc., v. 37, Jan. 1919 


"Communication Theory of Secrecy Systems," Bell System Technical Journal, 
v. 28, Oct. 1949 


"Programming a Computer for Playing Chess," IRE Proc., v. 37, Feb. 1919 
Review of book "Cybernetics" by N. Wiener, IRE Proc., v. 37, Nov. 1919 


"Synthesis of Two-Terminal Switching Circuits," Bell System Technical 
Journal, v. 28, Jan. 19h9 


Book review, "Transformations on Lattice and Structures of Logic," by 
Stephen A. Kiss, IRE Proc., v. 37, Oct. 19149 


"Simplified Derivation of Linear Least Square Smoothing and Prediction 
i IRE Proc., v. 38, Apr. 1950 (By H. W. Bode and C. E. 
Shannon 


"Chess-Playing Machine," Scientific American, v. 182, Feb. 1950 


"Communication Theory - Exposition of Fundamentals," Great Britain, Ministry 
of Supply. Symposium on Information Theory. Royal Society, 
Burlington House, London, Sept. 26-29, 1950 


"General Treatment of the Problem of Coding," Great Britain, Ministry of 


Supply. Symposium on Information Theory. Royal Society, Burlington 
House, London, Sept. 26-29, 1950 
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19. 


20. 
21. 
22. 


23. 
2h. 


25. 


26. 


Discussion of Dr. C. E. Shannon's papers (and reply), Great Britain, 
Ministry of Supply. Symposium on Information Theory. Royal 
Society, Purlington House, London, Sept. 26-29, 1950 

"Lattice Theory of Information," Great Britain, Ministry of Supply. 
Symposium on Information Theory. Royal Society, Burlington House, 
London, Sept. 26-29, 1950 


"Memory Requirements in a Telephone Exchange," Bell System Technical Journal, 
Ve 29, July 1950 


"Programming a Computer for Playing Chess," Phil. Mag., v. ll, Mar. 1950 
"Recent Developments in Communication Theory," Electronics, v. 32, Apr. 1950 


Review of book "Description of a Relay Calculator" by the Staff of the 
Harvard Computation Laboratory, IRE Proc., v. 38, Apr. 1950 


"A Symmetrical Notation for Numbers," Am. Math. Monthly, v. 57, Feb. 1950 


"Prediction and Entropy of Printed English," Bell System Technical Journal, 
Ve 30, Jan. 1951 


"The Mathematical Theory of Communication," University of Illinois Press, 
1919 (Book) (By C. E. Shannon and М. Weaver) 


"Computers and Automata," IRE Proc., v. lil, Oct. 1953 


